1. Overview of scenario data

First, I look at how participants rated the appropriateness of each violation in each scenario, and the appropriateness of the different types of punishments in that scenario.

Data is displayed separately by treatment, i.e. whether the violation is done by a friend/non-friend, and weather the punishment is performed by a friend to a friend/non-friend. Treatments are between subjects.

Insights:
All violations are considered fairly inappropriate, with insulting a family member and stealing being the worst.
There is quite some variation in appropriateness of the different types of punishment.
There are no striking differences between treatments.

Stealing

Phone

Interrupt

Insult

Avoid

1.1. Correlation of appropriantess of violations and appropriateness of punishment.

Correlation between violation ratings and punishment ratings across scenarios for each punishment type. The expectation is: negative relationship for angry remark, avoidance and gossip, and positive for doing nothing . (cf Fig. 1 of Eriksonn 2021)

Rho are Spearman ranked correlation coefficients (calculated across treatments). For a full overview see the table (third panel)

Without separating by domain

Separating by domain

full table of correlation coefficients

punishment_type domain treatment spearman_rho n
agry_remark insult nf -0.22 244
agry_remark insult f -0.16 239
agry_remark interrupt nf -0.08 241
agry_remark interrupt f -0.18 245
agry_remark online nf -0.23 240
agry_remark online f -0.10 237
agry_remark stealing nf -0.10 248
agry_remark stealing f -0.10 248
agry_remark texting nf -0.17 245
agry_remark texting f -0.02 244
avoid insult nf -0.10 243
avoid insult f -0.05 239
avoid interrupt nf 0.12 241
avoid interrupt f -0.07 245
avoid online nf -0.14 240
avoid online f 0.23 238
avoid stealing nf -0.13 248
avoid stealing f -0.08 248
avoid texting nf -0.13 245
avoid texting f -0.07 244
do_nothing insult nf 0.35 244
do_nothing insult f 0.23 239
do_nothing interrupt nf 0.12 242
do_nothing interrupt f 0.16 245
do_nothing online nf 0.31 240
do_nothing online f 0.15 237
do_nothing stealing nf 0.33 248
do_nothing stealing f 0.31 248
do_nothing texting nf 0.21 245
do_nothing texting f 0.39 244
gossip insult nf 0.02 244
gossip insult f -0.03 239
gossip interrupt nf 0.13 242
gossip interrupt f 0.07 244
gossip online nf 0.08 240
gossip online f 0.14 238
gossip stealing nf 0.11 248
gossip stealing f 0.08 248
gossip texting nf -0.02 245
gossip texting f 0.09 244

2. Analysis of appropriateness of violations across all domains

Now I merge all the scenarios together.

Insight:

Overall appropriateness is quite low, with again no striking difference between treatments.

2.1 ANOVA

First I run an ANOVA to get an idea of differences across treatments and domains. The ANOVA showed that both treatment and domain significantly affected ratings, with a significant interaction indicating that the effect of treatment varied across domains.

##                    Df Sum Sq Mean Sq F value   Pr(>F)    
## treatment           1     12   12.50   9.080  0.00261 ** 
## domain              4    324   80.97  58.817  < 2e-16 ***
## treatment:domain    4     47   11.71   8.507 8.16e-07 ***
## Residuals        2433   3349    1.38                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.2 Regressions

I now run two multilevel linear regressions with participants as random intercept to zoom in on size and direction of efffects.

First, a linear regression predicting rating ~ treatment shows a small effect of treatment: violations by friends are considered slightly more appropriate.

If we include a test of interactions between treatment and domain, we can see that the effect is driven by the “exclude from online chat scenario:

“One student who is part of your friends/others group (Student A) excluded another student, who is also a part of your friends group, from a group chat on WhatsApp.”

Rating ~ Treatment

  rating
Predictors Estimates CI p
(Intercept) 1.08 1.00 – 1.16 <0.001
treatmentf 0.14 0.05 – 0.23 0.003
Random Effects
σ2 1.26
τ00 ID2 0.27
ICC 0.18
N ID2 496
Observations 2443
Marginal R2 / Conditional R2 0.003 / 0.178

Rating ~ Treatment * Domain

  rating
Predictors Estimates CI p
(Intercept) 0.61 0.46 – 0.75 <0.001
treatmentf -0.04 -0.25 – 0.17 0.721
domain interrupt 0.53 0.32 – 0.74 <0.001
domain [online] 0.68 0.50 – 0.87 <0.001
domain stealing 0.35 0.14 – 0.56 0.001
domain [texting] 0.79 0.60 – 0.97 <0.001
treatmentf × domain
interrupt
0.13 -0.19 – 0.46 0.421
treatmentf × domain
[online]
0.74 0.48 – 1.00 <0.001
treatmentf × domain
stealing
-0.01 -0.33 – 0.32 0.970
treatmentf × domain
[texting]
0.07 -0.19 – 0.33 0.602
Random Effects
σ2 1.07
τ00 ID2 0.31
ICC 0.22
N ID2 496
Observations 2443
Marginal R2 / Conditional R2 0.103 / 0.301

3. Analysis of appropriateness of punishment if friend vs. non-friend

Now I look at the appropriateness ratings of punishments if the violator is a friend vs. non friend.

3.1 Vizualization

Again, visual inspection does not reveal any striking differences bewteen treatments and domains.

Overall appropriatess across domains

Appropriateness by domain

Appropriateness of punishment by treatment and punisment type

3.1 ANOVA

First, I run an ANOVA to get an idea of differences in mean ratings across treatments and domains.

There was no significant main effect of treatment.

In contrast, domain was significant, suggesting that the ratings varied meaningfully across different domains.

The interaction between treatment and domain was also significant, implying that the effect of treatment differed depending on the domain.

##                    Df Sum Sq Mean Sq F value Pr(>F)    
## treatment           1      0    0.00   0.002 0.9611    
## domain              4    475  118.69  68.455 <2e-16 ***
## treatment:domain    4     15    3.83   2.208 0.0656 .  
## Residuals        7284  12630    1.73                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

3.2 Regressions

Now I fit 3 linear regressions to better explore directions of effects.

First, only with treatment as independent variable, then I add interactions with domain (insult as reference) and finally I also include punishment type as exploratory variable (angry_remark as reference).

The main results are [see tabs below for full results]:

Model 1: no effect

Model2 :

  • There are differences between baseline ratings across domains, but no interactions {.underline}

Model 3:

  • The domain “texting” was associated with significantly lower ratings compared to the reference domain.

There a few significant interactions:

  • The punishment type gsip has lower ratings than the reference group.

  • The combination of “interrupt” domain and “avoid” punishment type was associated with lower ratings compared to the reference groups.

  • The combination of “stealing” domain and “avoid” punishment type was associated with higher ratings compared to the reference groups.

  • The combination of “texting” domain and “gossip” punishment type was associated with higher ratings compared to the reference groups.

  • The combination of “stealing” domain and “gossip” punishment type was also associated with higher ratings compared to the reference groups.

Rating ~ Treatment

  rating
Predictors Estimates CI p
(Intercept) 2.54 2.48 – 2.60 <0.001
treatmentf -0.01 -0.06 – 0.05 0.833
Random Effects
σ2 1.56
τ00 ID2 0.24
ICC 0.13
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.000 / 0.131

Rating ~ Treatment * Domain

  rating
Predictors Estimates CI p
(Intercept) 2.61 2.50 – 2.72 <0.001
treatmentf 0.01 -0.15 – 0.16 0.936
domain interrupt -0.20 -0.35 – -0.04 0.012
domain [online] -0.14 -0.27 – -0.01 0.030
domain stealing 0.36 0.21 – 0.52 <0.001
domain [texting] -0.41 -0.53 – -0.28 <0.001
treatmentf × domain
interrupt
-0.01 -0.26 – 0.23 0.907
treatmentf × domain
[online]
0.16 -0.02 – 0.34 0.078
treatmentf × domain
stealing
-0.09 -0.33 – 0.16 0.491
treatmentf × domain
[texting]
-0.09 -0.27 – 0.09 0.320
Random Effects
σ2 1.49
τ00 ID2 0.24
ICC 0.14
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.037 / 0.171

Rating ~ Treatment * Domain* punishment type

  rating
Predictors Estimates CI p
(Intercept) 2.89 2.73 – 3.05 <0.001
treatmentf -0.11 -0.34 – 0.12 0.361
domain interrupt -0.11 -0.34 – 0.12 0.355
domain [online] -0.16 -0.38 – 0.05 0.131
domain stealing 0.14 -0.09 – 0.37 0.231
domain [texting] -0.50 -0.71 – -0.28 <0.001
punishment type avoid -0.15 -0.36 – 0.07 0.175
punishment type [gossip] -0.69 -0.91 – -0.48 <0.001
treatmentf × domain
interrupt
0.17 -0.18 – 0.52 0.329
treatmentf × domain
[online]
0.27 -0.04 – 0.57 0.084
treatmentf × domain
stealing
0.09 -0.26 – 0.44 0.605
treatmentf × domain
[texting]
0.04 -0.27 – 0.34 0.816
treatmentf × punishment
type avoid
0.17 -0.13 – 0.47 0.264
treatmentf × punishment
type [gossip]
0.17 -0.13 – 0.47 0.271
domain interrupt ×
punishment type avoid
-0.41 -0.71 – -0.11 0.007
domain [online] ×
punishment type avoid
-0.08 -0.38 – 0.22 0.614
domain stealing ×
punishment type avoid
0.31 0.01 – 0.61 0.041
domain [texting] ×
punishment type avoid
-0.04 -0.34 – 0.26 0.813
domain interrupt ×
punishment type [gossip]
0.15 -0.15 – 0.45 0.325
domain [online] ×
punishment type [gossip]
0.15 -0.15 – 0.45 0.327
domain stealing ×
punishment type [gossip]
0.36 0.06 – 0.66 0.018
domain [texting] ×
punishment type [gossip]
0.30 0.00 – 0.60 0.050
(treatmentf × domain
interrupt) × punishment
type avoid
-0.22 -0.65 – 0.20 0.303
(treatmentf × domain
[online]) × punishment
type avoid
-0.21 -0.64 – 0.22 0.338
(treatmentf × domain
stealing) × punishment
type avoid
-0.22 -0.65 – 0.20 0.300
(treatmentf × domain
[texting]) × punishment
type avoid
-0.10 -0.52 – 0.33 0.661
(treatmentf × domain
interrupt) × punishment
type [gossip]
-0.34 -0.77 – 0.08 0.115
(treatmentf × domain
[online]) × punishment
type [gossip]
-0.11 -0.54 – 0.32 0.619
(treatmentf × domain
stealing) × punishment
type [gossip]
-0.31 -0.74 – 0.11 0.152
(treatmentf × domain
[texting]) × punishment
type [gossip]
-0.28 -0.71 – 0.14 0.195
Random Effects
σ2 1.44
τ00 ID2 0.24
ICC 0.15
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.070 / 0.205